To make someone annoyed or angry: “Gavin may seem unflappable, but I know a way to get his goat.” This expression comes from a tradition in horse racing. Thought to have a calming effect on high-strung thoroughbreds, a goat was placed in the horse's stall on the night before the race. Unscrupulous opponents would then steal the goat in an effort to upset the horse and cause it to lose the race.
To annoy: His bitching gets my goat sometimes
[1910+; perhaps fr depriving a racehorse of its goat mascot; perhaps fr French prendre sa che`vre, ''take one's source of milk or nourishment'']